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Abstract 

The stress loss function in metric multidimensional scaling is differentiable at local 
minima. In this note we generalize this result to more general functions of the distances. 
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version, the bib hie, and the complete Rmd hie. 


1 Introduction 


In multidimensional scaling we minimize the stress measure proposed by Kruskal (1964a), 
Kruskal (1964b) and defined as 


m _ 

<r ( x ) = ~ y x'A k xf 


k= 1 


( 1 ) 


The Wk are known positive weights , the 5k are known positive dissimilarities. The Ak are 
known positive semi-dehnite matrices. In multidimensional scaling the Ak are dehned in 
such a way that the x'AkX are squared Euclidean distances between n points in M p , but this 
results in this note are valid for any positive semic-dehnite Ak- 


Clearly stress is not differentiable at those x for which one or more of the x'AkX are zero. 
This can create problem for gradient-based algorithms, because the relevant derivatives may 
not exist and/or they may not vanish at stationary points. 


It is shown, however, in De Leeuw (1984) that the directional derivative 

_ . , a(x + ey) — cr(x) 

D a(x, y ) = lim ^^ 

4-0 6 


( 2 ) 


exists for all x and y. At a local minimum, i.e. an x for which Dcr(x, y) > 0 for each y, De 
Leeuw (1984), and more generally De Leeuw (2018), shows that x'AkX > 0, which implies 
that stress is differentiable at local minima. 
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2 Results 


We now generalize the result from De Leeuw (1984) to a much more general class of functions 
of the y/x'A k x. 

Theorem 1: [Differentiability] Suppose cr(x) = F(y/x'AiX, • • • , y/x'A m x) with F contin¬ 
uously differentiable and V k F(z) < 0 if z k = 0. Then o is differentiable at local minima, and 
at local minima x' A k x > 0. 

Proof: 

For the y/x'A k x we have 


\!ix + ey)'A k (x + ey) = \jx'A k x + 


f i 

J y/x'A k X 

\Vl/A k y 


x'A k y 


if x'A k x > 0 
if x'A k x = 0 


+ o(e). (3) 


Thus we find for the directional derivative of a 


D a(x,y) 


Y ‘DkF^x'Aix, • • • , \Jx'A m x) 

x , Af c x> 0 


l 

\Jx' A k x 


x'A k y+ 


V k F(yJx'Aix, • • • , \Jx'A m x)\Jy'A k y. 

x , Af t x=0 


( 4 ) 


If the assumption of the theorem is true there is an y such that the second term on the right 
in (4) is negative. If for that y the first term is positive we change y to —y. The first term 
becomes negative, the second term remains negative. Thus the derivative in the direction y 
is negative and x cannot be a local minimum. ■ 

Corollary 1: [Separable] Suppose a(x) = Y^ k =i fk(V x> A k x ) with the f k continuously 
differentiable and Df k ( 0) < 0. Then o is differentiable at local minima, and at local minima 
x'A k x > 0. 

Proof: In this case 

Y T> k F(y/x’A 1 x, ■ ■ ■ ,y/x'A m x)^/y'A k y = Y V A-(°)VV A k y, 

x'AkX =0 x'AkX=0 

which can be made negative for some y. ■ 

Corollary 2: [Least Squares] Suppose a(x) = Y^k=\ w k{^k ~ A k x)) 2 with (f) continu¬ 

ously differentiable, 0(0) = 0, and X>0(0) > 0. Then a is differentiable at local minima, and 
at local minima x'A k x > 0. 

Proof: In this case 

Y VkF^x'Axx, • • • , \Jx'A m x)yJy'A k y = - Y Wk^kD^)\Jy'A k y, 

x'Ai~x=0 x'A]^x =0 
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which can be made negative for some y. ■ 

Remark 1: [RStress] The result of corollary 2 can be applied to rStress (De Leeuw, 
Groenen, and Mair (2016)), defined as 


<j r (x = 


m 

E 

k =1 


w k (5 k - (■ \Jx'A k x) 7 


with r > 0. Thus <f>(z) = z r and T><p(z) = rz r ~ 1 . But if r < 1 then is not differentiable 
at zero and our results do not apply. If r > 1 then T>(f)(0) = 0, rStress is differentiable 
everywhere, and again our results do not apply (although obviously if r > 1 then rStress is 
differentiable at local minima, even if x'A k x = 0). The only rStress covered by our result is 
Kruskal’s stress, with r = 1. 
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